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1. 


INTRODUCTION 


Project  HINDSKfflT  is  a  study  of  the  recent  science  and  technology  \diich 
has  been  Utilized  by  the  Department  of  Defense  in  weapon  systems.  As  a  con¬ 
sequence,  the  focus  is  primarily  on  the  physical  and  engineering  sciences 
and  related  technologies.  Consideration  of  contributions  from  the  life, 
behavioral,  or  environmental  sciences  is  peripheral. 

The  study  Ir  retrospective  in  character  because  the  criteria  is  proven 
utilization  and  it  is  likely  that  delays  of  5  to  10  years,  or  more,  fre¬ 
quently  occur  between  a  scientific  discovery  or  an  invention  and  the  time 
of  its  practical  utilization.  The  effort  has  two  basic  objectives:  (l)  to 
identify  and  firmly  establish  management  factors  for  research  and  technology 
programs  which  have  been  associated  with  the  utilization  of  the  results  pro¬ 
duced  by  these  programs;  and  (s)  to  measure  the  overall  increase  in  cost- 
effectiveness  in  the  current  generation  of  weapon  systems  compeared  to  their 
predecessors  (when  such  can  be  identified)  which  is  assignable  to  any  part 
of  the  total  DOD  investment  in  research  and  technology. 

The  project  has  been  underway  for  2^  years  and  is  planned  to  continue. 
During  the  first  1^  years,  pilot  studies  developed  and  established  the 
techniques  of  analysis.  Throughout  the  past  year,  teams  of  in-house 
scientists  and  engineers,  working  with  the  voluntary  support  of  contrac¬ 
tors,  performed  the  bulk  of  the  data  collection.  We  estimate  that  about 
liO  pro.’essional  man-years  were  usea  in  collecting  and  analyzing  the  data 
reported  here. 

2.  THE  METHOD  OF  ANALYSIS 

The  method  of  analysis  has  the  following  steps: 

a.  A  recent  weapon  system  or  end-item  equipment  which  is  either 
already  in  the  inventory  or  is  committed  to  the  inventory  is 
selected  for  study. 

b.  A  team  of  5  to  10  scientists  and  engineers  selected  for  their 
expertise  in  the  area  of  the  subject  system  Is  appointed. 

c.  The  team  dissects  the  system  into  its  subsystems  and  components 
to  assure  systematic  analysis,  and  then  identifies  each  contri¬ 
bution  from  recent  science  and  technology  which,  in  their  judg¬ 
ment,  is  clearly  important  either  to  Increased  system  perfommnce 
or  to  reduced  cost,  compared  to  a  predecessor  system  when  such 
can  be  identified.  For  those  cases  where  no  predecessor  exists, 
the  contributions  (primarily  19^5  to  date)  are  those  which  are 
judged  to  be  essential  to  the  attainment  of  a  successful  system. 

In  making  these  judgments,  the  analyzing  team  worked  with  the 
principal  contractors  Involved  in  the  engineering  development 
and  production  of  the  system. 
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An  exaDjple  of  such  an  identified  contribution  would  be  the 
develoianent  starting  in  19^9  of  the  titanium- aluminum- vanadium 
alloy  used  in  the  compressor  blades  of  the  turbo- fan  engine  in 
the  C-l4l  transport  aircraft.  The  hi^  and  uniform  strength-to- 
weig^t  ratio,  the  corrosion  and  erosion  resistance,  the  notch- 
tou^iness  and  creep  resistance  of  this  material  substantially 
increased  the  efficiency  and  reliability,  and  reduced  the  wei^it 
and  extended  the  life  of  the  engine  con^ared  to  what  it  would 
have  been  had  one  used  the  steel  blades  eaployed  in  the  turbine 
engine  ^ich  drives  the  propellers  on  the  C-I3OA  aircraft  (the 
predecessor  system). 

d.  Once  a  contribution  has  been  judged  to  be  significant,  it  is  assigned 
to  one  of  the  team  members  for  further  study.  He  identifies  the  in¬ 
dividuals  who  were  the  principal  contributors,  the  organizations  with 
which  they  were  working  at  the  time  the  work  was  done,  the  date  when 
the  feasibility  or  practicability  of  the  idea  was  first  demonstrated, 
the  nature  of  the  work  (science  or  technology),  the  objective  of  the 
work,  the  approximate  cost,  the  funding  sources,  etc.  This  material 
is  written  up  in  a  uniform  format  and  becomes  a  part  of  our  per¬ 
manent  data  bank  of  utilized  science  and  technology.  We  believe 
we  have  demonstrated  that,  with  reasonable  effort,  one  can  trace 
the  origin  of  a  contribution  to  a  specific  time  and  to  particular 
people.  Consider  again  the  exan^jle  of  the  titanium  alloy  used 
in  compressor  blades,  mentioned  above.  The  development  of  this 
alloy  was  the  result  of  the  efforts  of  individuals  in  three  organi¬ 
zations.  Some  of  the  basic  work  was  done  in  19^+9  and  1950  at  the 
Armour  Eesearch  Foundation,  supported  by  the  Army  and  the  Air  Force, 
for  military  applications.  At  about  the  same  time,  further  work  was 
performed  at  the  Battelle  Memorial  Institute  funded  partly  by  the 
Air  Force  and  partly  by  industry  (the  Remcru  Titanium  Corporation). 

Over  the  next  10  years,  Remcru  and  others  carried  the  alloy  de¬ 
veloped  for  them  at  Battelle  into  production  and  thus  it  was  avail¬ 
able  for  use  in  the  Pratt  and  I’liltney  turbo- fan  engine  used  in  the 
C-l4l,  \diere  we  identified  it.  Ibis  material  is  known  to  have  many 
other  military  and  commercial  applications  as  well. 

It  is  important  to  note  that  this  technique  is  selective  in  identifying 
the  contributions  from  recent  science  and  technology.  The  many  Important 
contributions  which  pre-date  the  World  War  II  period  are  not  included;  nor 
are  the  countless  results  of  research  that,  althou^  indistinguishable  in 
themselves,  contribute  to  the  pool  of  general  knowledge  of  scientists  and 
engineers  from  which  ideas  are  drawn. 
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3.  DEFINITIONS 


Each  discrete,  identified  contribution  is  called  an  Event.  The  dura¬ 
tion  of  an  Event  includes  the  period  from  the  conceiving  of  an  idea  to 
the  initial  demonstration  of  feasibility  or  validity.  The  date  of  an 
Evei'it  is  the  end  of  this  period. 

In  order  to  provide  uniformity  of  reporting  and  to  facilitate  analysis 
of  the  data,  the  following  definitions  were  established  for  the  classifi¬ 
cation  of  research  activities  and  lor  the  ascribing  of  motivation  to  the 
identified  contributors. 

Science;  Theoretical  and/or  experimental  studies  of  new  or  une:q)lored 
natural  phenomena. 

Science  Events  are  divided  into  two  categories  which  are  de¬ 
termined  by  the  objective  of  the  work: 

(a)  Undirected  Science,  in  which  the  objective  of  the  work 

is  the  advancement  of  knowledge  for  its  own  sake,  without 
regard  for  possible  application,  and 

(b)  Applied  or  Directed  Science,  in  which  the  objective  of  the 
work  is  to  produce  an  understanding  of  phenomena  or  speci¬ 
fic  knowledge  which  is  needed  for  some  particular  applica¬ 
tion  or  class  of  applications. 

Technology;  The  conception  and/or  demonstration  of  the  capability  of 
performing  a  specific  elementary  function  using  new  or  un¬ 
tried  concepts,  principles,  techniques,  or  materials,  or 

The  measurement  of  the  behavior  of  materials  and  equipment  as 
required  for  design,  or 

The  first  demonstration  of  the  capability  of  performing  a 
specific  elementary  function,  using  established  concepts, 
principles,  or  materials,  or 

Tlie  development  of  new  manufacturing,  fabrication  or  processing 

techniques. 

The  definitions  of  science  make  the  work  substantially  synonymous  with 
the  current  D(D  category  of  Research  (6.1),  and  the  definitions  of  tech¬ 
nology  make  the  work  substantially  synonymous  with  the  current  DQD  category 
of  Ekploratory  Development  (6.2). 

The  tens  "recent”  or  "new”,  as  applied  to  science  or  technology, 
generally  loplies  "post  194^”.  However,  if  an  item  of  science  or  tech¬ 
nology  was  known  to  have  been  used  in  a  predecessor  system,  "recent” 
implies  an  Incremental  contribution  to  that  science  or  technology  after 
the  engineering  design  date  of  the  predecessor. 
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4.  THE  SYSTEMS  STUDIED 


Figure  1  lists  the  20  systems  studied,  the  number  of  Events  vhich  have 
been  fully  analyzed  and  are  included  in  the  present  data  beink,  and  indi¬ 
cates  the  predecessor  system,  if  any.  15  of  the  20  systems  contributed  a 
total  of  638  Events,  of  idiich  556  are  distinct.  That  is,  82  Events  were 
identified  more  than  uue  system. 

The  selected  systems  represent  a  fairly  balanced  sample  of  different 
types  of  military  equipment. 

The  list  of  Events  for  each  system  is  not  exhaustive,  but  rather  con¬ 
tains  those  Events  irtiich  the  team  Judged  to  be  most  important.  The  teams 
estimate  that  they  studied  between  30^  and  of  the  Events  (depending 
on  the  system)  ^ich,  in  a  more  extensive  study,  would  be  found  worthy 
of  analysis. 

We  have  considerable  confidence  in  the  present  sample  of  556  Events 
since  the  properties  of  the  sample  have  not  changed  substantially  from 
the  time  that  the  data  base  had  less  than  100  Events  taken  from  7  systems. 

5.  PRINCIPAL  FINDINGS 

Based  upon  556  documented  Events  in  15  weapon  systems,  we  state  the 
following  findings: 

a.  The  Number  of  Events  Per  System  -  A  large  number  of  significant 
science  and  technology  Events  (50  to  100  or  more)  is  readily 
identified  as  being  utilized  in  the  larger  systems  (reference 
Figure  1). 

b.  The  Importance  of  the  Events  -  In  the  Judgment  the  technical 
analysts,  it  is  the  synergistic  effect  of  these  many  Events 
which  is  the  primary  source  of  the  incredse  in  performance  or 
the  reduction  in  procurement  cost  of  the  system  compared  to 
its  predecessor.  Without  most  of  the  other  Events  (or  their 
equivalent),  any  one  Event  or  even  a  small  number,  would  make 
little,  if  any,  contribution  to  a  particular  system. 

c.  The  Time  Distribution  of  Events  -  For  those  systems  which  have 
an  identlflabie  predecessor  (8  systems,  75?^  of  all  Events),  the 
great  majority  of  the  Events  utilized  did  not  exist  at  the  time 
the  predecessor  system  was  committed  to  engineering  develo|»nent . 
This  situation  can  be  seen  in  the  Figures  referenced  in  the 
following  paragraphs. 

Figure  2  shows  the  time  distribution  of  the  Events  in  the  ME  k6 
torpedo.  The  early  stages  of  this  program  preceding  formal  de¬ 
velopment  were  guided  by  a  well  organized  technology  and  systi^- 
concept  program— the  RETORC  program.  Note  that  only  11^  of  the 
Events  Judged  essential  to  the  performance  of  the  MC  46  existed 
at  the  time  of  the  MK  44  prototype.  Note  also  that  in  spite  of 
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the  existence  of  a  well  organized  technology  and  system- concept 
program,  of  the  Events  occurred  after  the  system  design 
contract  was  signed — primarily  a  consequence  of  changing  speci¬ 
fications  from  the  Mod  0  to  the  Mod  1. 

Figure  3  shows  the  time  distribution  of  the  Events  in  the  MK  56-57 
mines.  This  mine  development  was  unusixal  in  that  it  had  an  extended 
stage  which  corresponds  to  what  we  now  know  as  advanced  d envelopment, 
culminating  in  a  pilot  production  of  over  100  working  models  which 
were  used  in  an  exten  ive  test  and  evaluation.  The  drawings  were 
"released  for  production"  in  1958^  tut  regular  production  was  not 
funded.  Between  1958  and  i960,  the  specifications  were  changed 
(the  required  maximum  operating  depth  was  doubled),  and  in  i960, 
a  production  contract  was  let  to  the  new  specifications.  Again 
we  see  the  consequence  of  a  change  in  specifications:  new  steels, 
new  methods  of  sealing  leads,  and  many  other  innovation  changes 
were  needed  to  make  the  system  useful,  and  largely  account  for 
the  19^  of  the  Events  which  followed  the  I958  release  date. 

Figure  4  shows  the  time  distribution  of  the  Events  in  LANCE.  For 
this  system,  the  percentage  of  Events  (42^)  following  the  engineer¬ 
ing  development  contract  date,  is  some'idiat  above  average.  In  this 
system,  no  c  T^ilete  working  model  had  been  made  before  commitment 
to  engineering  development.  Indeed,  in  some  cases,  inrpertant  sub¬ 
systems  had  not  been  assembled  or  flight  tested. 

Figure  5  shows  a  composite  distribution  of  all  systems  in  which 
the  year  zero  is  the  engineering  development,  system  design,  or 
production  contract  date,  whichever  came  first.  Note  that,  for 
the  8  systems  with  identified  predecessors,  the  average  prede¬ 
cessor  date  was  earlier  by  I3  years.  We  see  that  only  11^  of 
edl  the  Events  occurred  13  or  more  years  before  the  system  being 
analyzed  was  designed.  Of  the  Events  in  this  11^,  however,  many 
were  not  really  available  to  a  predecessor  system  Inasmuch  as  the 
date  of  an  Event  is  defined  as  the  "demonstration  of  feasibility" 
and,  in  most  cases,  considerable  further  work  is  needed  before 
the  system  engineer  can  confidently  include  the  results  into  an 
engineering  design. 

We  note  that  3Tlt  of  the  Events  occurred  after  the  system  was  com¬ 
mitted  either  to  engineering  design  or  to  production— a  fair  number 
occurring  even  U  or  5  years  later. 

The  time-normalized  Event  distribution  plot  of  Figure  3  has  an 
Interesting  form  which  Is  probably  significant  to  understanding 
the  RDT&E  process.  The  Events  steurt  accumulating  about  20  years 
before  the  system  design  date,  at  a  rate  which  Increases  steadily. 
About  2  years  before  the  system  design  date, -the  rate  of  Event 
production  nearly  doubles.  Following  the  design  date,  the  rate 
drops  fairly  rapidly,  reaching  a  negligible  level  on]^  after  6  or 
7  years. 
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d.  Performing  Organizations  -  Eie  556  Events  were  performed  by  approx! 
mately  300  different  organizations,  sometimes  alone  and  sometimes 
In  a  joint  effort,  In  which  case  credit  was  shared  between  the  or¬ 
ganizations.  Oiir  file  contains  a  list  of  1,025  people  listed  as 
the  principal  contributors. 

We  find  (see  Figure  6)  that  over  the  entire  period  covered  by 
the  study,  39^^  of  the  Events  were  performed  in  DOD  in-house 
laboratories,  U956  In  industry,  9^  in  universities,  including 
the  research  contract  centers,  2^  was  performed  in  non-DOD 
Federal  laboratories,  and  less  than  1^  in  foreign  laboratories. 

It  is  clear  that  althou^  the  in-house  laboratories  have  made  a 
substantial  contribution  the  relative  iii5)ortance  of  the  in-house 
laboratory  has  been  declining.  As  Figure  7  shows,  the  proportion 
of  Events  coming  from  the  in-house  laboratories  has  decreased 
gradually  from  about  60^  in  the  early  19^^•0's  to  about  30^^  in  the 
last  few  years.  However,  this  is  not  necessarily  to  the  discredit 
of  the  laboratories.  During  the  20- year  period  covered  by  the 
study  the  national  scientific  and  technological  community  essen¬ 
tially  quadrupled  in  strength  while  the  in-house  laboratories 
were  constrained  to  a  growth  of  about  a  factor  or  two,  Hie  con¬ 
sequence  of  the  in-house  growth  is  reflected  in  Figure  8  which 
shows  that  there  has  been  about  a  doubling  of  Events  produced 
per  year  in-house  as  the  professional  manpower  increased. 

The  university  contribution  was  approximately  constant  during 
this  period  and,  therefore,  industry  has  made  the  relative 
increase.  This  is  consistent  with  the  fact  that  industry  has 
enjoyed  the  greatest  increase  in  funding  and  in  relative  funding. 

e.  Funding  -  An  examination  of  the  funding  source  for  each  Event 
shows  (^see  Figure  9)  that  DOD  direct  funding  was  by  far  the 
largest  source  {Q'Jia  of  the  Events),  ’defense  industry  (either 
from  profits  or  from  IR&D  allowable  overhead  costs)  funded  9^. 

Thus,  one  can  say  that  in  a  very  real  sense  96^  of  the  E-rents 
were  fimded  by  dollars  appropriated  for  purposes  of  defense. 

We  were  surprised  to  find  that  non- defense  industry  funded  only 
2^  of  the  Events.  Further,  most  of  these  are  the  series  of  8 
Events  performed  at  the  Bell  Telephone  Laboratories  on  the  early 
development  of  the  transistor.  We  have  classed  this  organization 
as  comserclally-orlented,  although  for  many  years  It  has  also  been 
one  of  the  largest  defense  contractors.  The  transistor  Events 
have  probably  significantly  affected  more  types  of  defense  equip¬ 
ment  than  any  other  Event  or  set  of  related  Events. 

Another  Event  of  widespread  ioqportance  to  defense  which  we  Include 
In  the  Industrially  funded  category  Is  the  welding  of  aluminum. 

(The  bcurrel  and  chemical  container  Industry  was  primarily  Involved. 
A  close  examination  of  this  Event  discloses,  however,  that  In  the 
early  stages.  It  received  substantial  support  fron  the  Manhattan 
District  and  also  from  the  Navy. 
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These  exajJ5)les  illustrate  the  fact  that  the  value  pf  Events  to 
defense  may  differ  widely.  Thus,  simply  counting  the  number  of 
Events  in  a  particular  category  is  only  a  first-order  approxi¬ 
mation  to  the  value  attributed  to  that  category. 

f.  Average  Cost  Per  Event  -  As  we  have  already  noted,  an  Event  is 
regarded  as  completed  at  the  time  that  feasibility  is  first  demon¬ 
strated.  By  this  definition,  the  median  cost  of  an  Event  was 
found  to  be  about  $40,000.  However,  there  is  almost  inevitably 

a  subsequent  cost  to  carry  the  idea  to  the  point  \^ere  it  can  be 
used  with  confidence  in  engineering  design  and  production.  In 
a  small  number  of  cases  (15)>  we  have  been  able  to  also  estimate 
this  subsequent  cost  and  have  found  that  it  usually  varies  from  10 
to  20  times  the  initial  cost.  In  the  titanium  alloy  example,  the 
initial  stage  of  the  Event  at  Armour  Research  Foundation  cost  about 
$50,000  but  the  subsequent  development  (industrially  financed,  in 
this  case)  cost  well  over  $500,000. 

g.  The  Objective  of  the  Work  Leading  to  an  Event  -  Finally  we  turn 
tc  a  particularly  important  aspect  of  the  environment  surrounding 
the  production  of  an  Event:  We  ask,  \rtiat  was  the  objective? 

This  is  an  important  question  since  it  gets  at  the  process  by 
>diich  science  and  technology  are  "coupled"  to  application.  We 
believe  the  best  (and  probably  the  only)  way  to  get  at  the  ob¬ 
jective  is  to  personally  query  the  key  individuals  who  either 
performed  the  work  or  were  closely  involved  in  its  supervision. 

In  90^  of  our  data  base,  we  have  been  able  to  Identify  the  objec¬ 
tive  or  motivation  as  it  was  perceived  by  the  participants. 

Before  discussing  the  data  relating  to  the  Objectives,  we  find 
that  it  is  desirable  to  first  categorize  the  Events  as  either 
Science  or  Technology.  This  matter  can  be  determined  fairly 
objectively  from  the  nature  of  the  work.  In  the  Science  cate¬ 
gory,  the  primary  result  is  new  knowledge  and  is  similar  to  the 
work  supported  in  the  present  DOD  catej^ry  of  Research  (6.1). 

In  the  Technology  category,  the  primary  result  is  an  iaproved 
technique,  material,  component  or  subsystem,  and  is  characteristic 
of  the  work  supported  in  the  present  DOD  category  of  Esqploratory 
Development  (6.2).  In  the  past,  and  even  in  the  present,  much 
work  of  this  type  is  associated  and  funded  integrally  with  the 
system  development  process.  In  this  case,  it  is  commonly  known 
as  Supporting  Technology.  However,  in  categorizing  Events  as 
Science  or  Technology,  it  is  the  nature  of  the  work  which  is  de¬ 
termining,  not  the  nature  of  the  funding  source. 

We  found  (see  Figure  10 )  that  Qi  of  the  Events  are  categorized 
as  Science  and  9^  as  Technology. 

Of  the  Science  Events,  the  great  majority  (6.2^  of  all  Events) 
were  applied  research,  clearly  oriented  toward  a  DOD  need.  Host 
of  the  balance  (1.51^  of  all  Events),  was  applied  research  with 
a  commercial  objective,  (ihese  were  the  8  Events  in  the  transistor 
sequence  at  the  Bell  Tele:^one  Laboratories.)  Only  two  Science 
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Events  (.31^  of  Events)  were  Identified  that  appeared  to  have 
a  minimum  relation  to  any  applied  objective.  They  were  the  early 
development  of  the  shock  tube  at  Cornell  University  and  a  project 
in  statistical  sen5)ling  at  Wayne  University.  Incidentally,  both 
of  these  Events  were  funded  by  ONE  in  the  late  19^*0' s. 

In  the  technology  category,  we  find  that  a  substantial  number  (25^ 
of  all  Events)  had  as  their  objective  the  development  or  advance¬ 
ment  of  "generic  DQD-oriented  technology."  This  is  technology 
aimed  at  a  broad  category  of  defense  needs  and  not  the  needs  of 
a  particular  system  or  system  concept,  such  as  higher  power  com¬ 
ponents  for  radars,  iaiproved  eotplosives,  more  efficient  solid 
propellants,  titanium  alloys,  and  general  purpose  electronic 
conponents.  We  find  that  very  frequently — in  about  755^  of  the 
cases— the  focus  and  spur, and  often  the  funding  is  provided  to 
the  technologists  by  one  or  more  systems  under  development, 
therefore,  we  could  reduce  the  figure  for  generic  technology 
by  1S$,  adding  it  to  the  advanced  development  and  system  develop¬ 
ment  categories  without  justifiable  challenge.  However,  the 
breakout  as  shown  is  Intended,  additionally,  to  offer  an  indica¬ 
tion  of  how  much  of  the  newly  developed  technology  is  inherently 
multi-useful. 

The  biggest  single  source  of  motivation  for  technology  innovations 
(U6it  of  all  Events)  is  a  system  or  system- concept  In  what  we  would 
now  identify  as  the  Advanced  Development  stage.  In  addition,  we 
Identify  l8^t  of  all  Events  as  being  motivated  by  a  DOD  system  in 
the  stage  we  now  call  Engineering  Development.  Thus,  a  DOD  sysem 
need  motivated  of  all  Events.* 

Finally,  we  note  that  only  3%  of  all  Events  were  motivated  by  a 
need  in  a  non- DOD  end  item. 

If  we  Identify  in  Figure  10  the  Events  which  were  motivated  by 
a  clearly  perceived  DOD  need,  ve  see  that  they  add  up  to  95%  of 
all  Events,  and  of  this  95%t  half  were  motivated  by  a  system  in 
>d)at  we  now  call  the  Advanced  Development  stage. 

h.  Heed-Recomition  and  Tectoical  Initiative  -  Since  it  became  clear 
early  in  &e  study  tkat  tecknoiogical  innovation  was  highly  corre¬ 
lated  with  need- recognition,  we  collected  additional  Information 
about  this  process.  In  particular,  we  separately  determined  for 
all  technology  Events  (9^  of  all  Events)  the  origins  of  the  nc;id* 
recognition  step  and  of  the  technical  solution  itself.  We  found 


*  T^ere  appears  to  ^  no  way  of  finding  out  with  any  accuracy  what  fraction 
of  funds  exi>ended  over  the  past  10  to  20  years  in  the  category  of 
science  and  technology  were  expended  under  the  condition  that  the  per- 
fonr'.ng  organisation  was  primarily  exposed  to  a  need  in  DCD  ^nerlc 
ted  oology,  a  need  of  a  system  in  the  advanced  development  stage  or  a 
net  i  of  a  system  in  the  engineering  development  stage.  Budget  categories 
are  not  a  reliable  Indicator  of  this  condition. 
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that  In  only  15?6  of  the  cases  the  need  was  first  recognized  by 
the  performing  research  groiq)  itself,  whereas  in  85^  of  the  cases 
the  need  was  first  recognized  either  by  an  applications  engineer¬ 
ing  group  or  an  external  group  associated  with  the  system  being 
considered  (a  "systems  group"). 

We  further  found  that,  where  the  need  was  first  recognized  by  the 
performing  research  group  (15^  of  the  cases),  this  group  was  in¬ 
variably  also  responsible  for  the  nature  of  the  technical  solution. 
However,  e»^n  in  those  cases  v^ere  the  need  was  first  recognized 
by  a  "systems  group"  (85^  of  the  cases),  the  initiative  for  the 
technical  solution  was  still  found  to  be  primarily  (72^  of  the 
time)  the  sole  responsibility  of  the  performing  research  group, 
and  in  the  balance  of  these  cases  the  technical  solution  was 
shared  by  the  research  group  and  the  "systems  group." 

Stating  the  same  figures  another  v&y,  we  see  that  the  nature  of 
the  technical  solution  was  determined  solely  by  the  performing 
research  group  76^  of  the  time  and  was  jointly  determined  with 
a  systems  group  2^^  of  the  time. 

i.  Summary  of  Findings 

(1)  Many  Events  (50  to  lOO)  \diich  are  innovations  in  science  or 
technology,  are  utilized  in  a  typical  advanced  system. 

(2)  These  Events  are  the  primary  source  of  the  increase  in  per¬ 
formance  or  the  reduction  in  cost  of  the  system  compared  to 
its  predecessor. 

(3)  The  Events  begin  accumulating  about  20  years' before  the  en¬ 
gineering  design  date  of  the  system  which  utilizes  them. 

The  rate  of  production  increases  steadily  to  a  peak  one  or 
two  years  before  the  system  design  date,  and  then  decreases 
gradually  reaching  zero  about  rlx  years  after  this  date. 

(^)  391^  vere  performed  by  In-kouse  laboratories,  by  industry, 

and  by  universities.  The  in-house  laboratory  contribution 
has  been  dropping  etead^'y  with  time  to  a  recent  level  of 
about  30^,  due,  at  leat  u  in  part,  to  reduced  relative  funding 
and  staffing. 

(5)  9^  of  the  Events  were  funded  by  defense  appropnati;»i6 
(directly  by  DC©  871t»  or  indirectly  by  Defense  Industry 

(6)  The  median  cost  per  Event  to  the  point  of  initial  desmm- 
stration  of  feasibility  war  about  |ii0,000.  Subsequent  costs, 
necessary  to  carry  the  Event  to  final  utilization,  appear  to 
be  10  to  20  times  Utis  amount. 

(7)  A  clear  understanding  of  a  D(S)  need  motivated  9^%  of  all 
Events.  (78^  of  all  Science  Events,  and  971*  of  all  Technology 
Events) 
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(8)  For  Technology  Events,  the  recognition  of  need  occurred 
primarily  in  a  "system  group"  of  Events),  vhereas  the 
technical  solution  itself  was  determined  primarily  by  the 
performing  research  organization  (75?^  of  Events). 

6.  CONCLUSIONS 


In  the  previous  section  we  have  presented  the  principal  facts  estab¬ 
lished  by  the  study  and  have  avoided,  as  much  as  possible,  the  drawing 
of  any  inferences.  We  now  wish  to  extend  the  discussion  to  include  cer¬ 
tain  conclusions  that  appear  to  be  well  supported  by  these  facts.  Before 
doing  so,  however,  it  is  necessary  to  make  two  caveats.  First,  the  simple 
counting  of  the  number  of  Events  in  a  specified  category,  as  thou^  all 
Events  had  equal  value,  allows  only  a  first-order  approximation  of  the 
true  value  of  the  sum.  The  recognition  of  this  fact  is  particularly  im¬ 
portant  in  an  attempt  to  make  precise  estimates  of  the  relative  value  of 
a  set  of  Events  prouuced  by  one  source  as  compared  to  a  set  of  Events 
produced  by  another  source. 

Second,  in  some  categories,  the  sample  size  is  quite  small  and  large 
statistical  errors  are  to  be  expected. 

With  these  reservations  in  mind,  we  can  proceed  to  an  intei^retation 
of  the  factual  findings. 

Several  significant  conclusions  regarding  the  system  development 
pr  cess  as  it  has  been  practiced  over  the  past  two  decades,  can  be  drawn 
from  an  examination  of  the  time  distribution  of  Events.  As  was  demon¬ 
strated  in  the  conqposite  time  distribution  of  Figure  5,  the  innovations 
for  a  given  system  begin  nearly  20  years  before  the  system  design  date, 
but  only  11^'  have  accumulated  at  the  13- year  mark,  when,  on  the  average 
d  predecessor  system  was  designed.  The  Events  are  produced  at  a  steadily 
Increasing  rate  up  until  a  year  or  two  before  the  system  design  date,  at 
^ich  time  the  rate  of  production  appears  to  increase  significantly. 
Following  the  design  date  the  Event  production  rate  drops  steadily  but 
does  not  reach  oero  until  about  six  years  have  elapsed. 

Our  first  conclusion  is  that  engineering  design  of  military 
wespon  systems  primarily  consists  of  sklTlfully  selecting  and  in¬ 
tegrating  a  large  number  of  innovations  so  as  to  produce,  by  syner¬ 
gistic  effects,  the  hl^  performance  demanded.  It  appears  to  us 
that  the  rate  of  accretion  of  useful  innovations  paces  the  time 
separation  between  successive  systems  with  a  given  performance 
differential.  A  utilized  innovation  can  occur  only  when  there  is 
a  conjunction  of  three  elements:  (l)  a  recognized  need;  (2)  com¬ 
petent  people  with  relevant  scientific  or  technological  ideas;  and 
(3)  financial  support.  A  deficiency  in  any  one  of  these  elements 
is  therefore  the  pacing  factor  in  the  production  of  any  given 
utilized  innovation. 


_Second,  ve  conclude  that  the  of  the  Events  which  occurre<1 
after  englnepring  design  was  Initiated  were  necessary  to  the  nitimn-ho 
performance  of  the  system.  The  process  of  selecting  and  integrating 
a  large  number  of  innovations  is  highly  creative  and  the  interactions 
of  the  many  innovations  are  frequently  very  complex  and  difficult  to 
anticipate.  We  believe  that  these  late- appearing  Events  vere  largely 
not  predictable  in  advance.  In  addition  to  those  attributable  to  the 
unexpected  interactions  between  innovations  some  were  due  to  new  in¬ 
sists  generated  in  the  intense  process  of  system  design  that  offered 
higher  performance  or  reduced  cost.  Still  others  were  the  consequence 
of  the  escalation  of  formal  requirements  or  specifications  in  the  ab¬ 
sence  of  a  proven  technological  base  for  their  attainment.  \'/hatever 
the  reason  it  appears  that  an  effort  to  actually  build  and  operate  a 
complete,  working  system  (not  just  produce  a  paper  design)  generates 
a  burst  of  innovative  activity. 

Third,  we  conclude  that  at  the  time  of  its  design  the  predecessor 
system  was  technolo^- limited.  This  follows  from  the  fact  that  on 
the  average  only  11^  of  the  science  and  technology  (Events)  utilized 
in  the  successor  system  existed  at  the  design  date  of  the  predecessor, 
and  many  of  these  were  not  yet  in  the  state  to  be  used  with  confidence 
in  engineering  design,  (in  some  cases  predecessor  systems  are  substan¬ 
tially  upgraded  over  their  initial  models  by  retrofitting  advanced  com¬ 
ponents  based  on  new  teclmclogy.  Thus,  in  our  example  to  be  discussed 
later,  the  C-I3O-E  (1964)  has  a  substantially  improved  performance  com¬ 
pared  to  the  C-I3O-A  (195^)>  but  is  still  inferior  to  the  C-141-A  (1964) 
which  used  even  more  new  technology  than  the  C-I3O-E.) 

Fourth,  we  conclude  that  the  relative  efficiency  of  production 
of  science  and  technology  Events  which  have  been  utilized  in  defense 
is  substantially  higher  when  funded  and  managed  by  the  Defense  Depart¬ 
ment  or  defense  industry  than  it  is  when  funded  and  managed  by  the  non¬ 
defense  sector  of  government  or  industry. 

We  draw  the  first  support  for  this  conclusion  from  the  funding 
data  in  Figure  9>  where  it  is  shown  that  the  defense  sector  funded 
96^  of  the  Events,  and  the  non-defense  sector  funded  only  4^  of 
the  Events.  We  estimate  that  from  1945  to  1963^  the  defense  sector 
expended  about  60^  of  the  U.S.  funds  in  the  category  of  science  and 
technology  and  the  non-defense  sector  about  40^.  Thus,  per  dollar 
inv'jsted  in  scientific  and  technological  effort,  the  defense  sector 
produces  many  more  defense-utilized  Events  than  does  the  non-defense 


sector.  In  particular,  the 

ratio. 

(defense-utilized  Events  produced) 

(defense- utilised  Events  produced) 

(defense- funded  and  managed 
research) 

(non-defense 

funded  and  managed 
research) 

is 

96?^  . 

H 

40^ 

or 

16  : 
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That  is,  science  and  technology  funds  deliberately  invested  and 
managed  for  defense  purposes  have  been  about. one  order  of  magnitude 
more  efficient  in  producing  useful  Events  than  the  same  amount  of 
funds  invested  without  specific  concern  for  defense  needs.  Thus, 
we  see  that  although  technological  "spin  off"  into  defense  weapon 
systems  from  the  non-defense  sector  exists,  it  is  very  small,  eind 
it  is  quite  inadequate  to  produce  the  n'jmber  of  innovations  needed 
to  make  possible  the  large  increases  in  performance  which  have  been 
attained. 

Our  second  support  for  this  conclusion  Is  drawn  from  data  on 
objective  or  motivation  of  Figure  10.  There,  we  noted  that  of  the 
utilized  science  Events,  75'i^  were  motivated  by  a  DOD  need  as  under¬ 
stood  by  the  priticipal  investigators,  and  of  the  utilized  technology 
Events,  97^  were  motivated  by  a  recognized  DOD  need.  It  appears 
that  the  great  majority  of  defense-utilized  Science  and  Technology 
Events  were  the  result  of  a  deliberate  and  successful  effort  to 
couple  the  Innovation  to  the  real  problems  of  defense. 

We  believe  that  to  a  considerable  degree  this  conclusion  is  the 
logical  consequence  of  the  first  conclusion  which  states  that  it  is 
tlie  synergistic  effect  of  many  innovations  which  produce  large  im¬ 
provements,  for,  in  general,  an  invention  has  to  "fit  in"  as  a  part 
of  a  system  or  system- concept  along  with  50  or  100  other  inventions. 
Hence,  innovators,  particularly  technology  innovators,  who  do  not 
understand  the  current  and  anticipated  needs,  are  less  likely  to 
produce  ideas  which  are  useful.  This  logical  inference  is  further 
supported  by  the  data  in  Figure  10.  There  we  observed  that  64^  of 
all  the  Events  had  as  motivation  a  DOD  system  or  system  concept. 


Further  insight  into  understanding  the  reason  why  Events  funded 
and  managed  by  defense  occur  with  such  a  relatively  hl(?h  frequency 
may  be  obtained  by  considering  the  data  in  Finding  (h).  There  we 
observed  that  for  technology  Events  need-recognition  occurred  within 
the  performing  research  organization  in  only  15^  of  the  cases.  In 
the  gi-eat  majority  of  technology  Events  (05^^),  it  required  a  "system 
group"  of  some  sort  to  first  perceive  and  state  the  need.  Thus, 
since  a  "systems  group"  for  a  defense  system  is  almost  invariably 
closely  associated  with  a  defense  program,  the  resulting  technology 
is  also  similarly  associated. 

Fifth,  we  conclude  that,  in  the  systems  we  studied,  the  contributions 
from  recent  (post  1945)  research  in  science  were  greatest  when  the  effort 
was  oriented.  In  order  to  examine  the  consequence  of  allowing  the  scien¬ 
tist  to  select  the  problem  area  to  be  investigated  as  an  alternative  to 
selection  of  the  area  by  management,  two  classes  were  defined  within  the 
study:  "Undirected  Research",  that  is,  nonoriented  research,  and  "Applied 
Research",  that  is,  research  directed  toward  the  timely  gaining  of  know¬ 
ledge  in  areas  of  specific  interest  to  the  sponsor.  As  used  here,  these 
ternis  are  meant  to  describe  the  motivation  of  the  research  scientist  rather 
than  the  nature  of  the  work  in  which  he  is  engaged.  For  example,  we  have 
classified  as  "Applied  Science"  a  very  fundamental  and  important  series  of 
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eight  transistor-associated  basic  scientific  events  occurring  at  the  Bell 
Telephone  Laboratories.  We  believe  that  these  events  are  classified  cor¬ 
rectly  because  they  occurred  in  a  mission-oriented  environment  and  were 
clearly  in  support  of  the  mission. 

There  is  no  question  that  over  a  long  time  scale  undirected  research 
has  had  great  value.  The  sequence  of  contributions  in  atomic  and  nuclear 
physics  culminating  in  the  discovery  of  fission  in  1939  has  had  a  revolu¬ 
tionary  impact  on  military  arms  and  strategy.  Without  the  organized  body 
of  physical  science  extant  in  1930  —  classical  mechanics,  quantum  mechanics, 
relativity,  thenucdynamics,  optics,  electromagnetic  theory  and  mathematics  — 
none  of  the  science  events,  and  only  a  fraction  of  the  technological  events 
could  h.Vve  occurred.  Thus,  in  the  past,  in  at  least  these  areas,  undirected 
research  has  paid  off  on  the  30  to  60  year  or  more  time  scale.  In  our  stiidy 
we  see  no  eirLdence  that  this  situation  has  changed.  However,  the  fact 
remains  that  the  contribution  from  recent  (essentially,  post  19U5)  undirected 
science  to  the  systems  we  have  studied  appears  to  have  been  small. 

We  emphasize  that  this  conclusion  does  not  question  the  value  of 
scientific  research.  (Recalling  Figure  10,  Q%  of  the  identified  Events 
were  scientific  in  nature.)  Instead,  it  focuses  on  the  relative  values 
of  alternative  practices  in  the  management  of  scientific  research  and 
suggests  that  the  length  of  time  to  utilization  of  scientific  findings 
is  decreased  when  the  scientist  is  working  in  areas  related  to  the  problems 
of  his  sponsor. 

We  believe  that  the  observation  regarding  the  relative  times  to  utili¬ 
zation  of  orietited  as  opposed  to  nonoriented  research  and  the  previous  con¬ 
clusion,  that  the  efficiency  of  production  of  defense  utilized  science  (and 
technology)  is  substantially  enhanced  when  funded  and  managed  by  the  Depart¬ 
ment  of  Defense,  cannot  be  disassociated.  Conjointly  they  demonstrate  that 
both  the  efficiency  of  productioii  and  the  timeliness  of  knowledge  useful  to 
a  mission-oriented  agency  are  most  readily  achieved  when  that  agency  funds 
and  manages  its  own  research  program. 

Sixth,  we  estimate  that  the  POD  investment  in  scie.’'ice  and  tech¬ 
nology  has  had  a  large  payoff,  although  delayed  in  time  by  3  to  10  years. 

To  support  this  conclusion  we  will  give  several  examples  where 
substantial  increases  in  cost-effectiveness  between  successive  gene¬ 
rations  of  weapon  systems  have  occurred,  attributable  primarily  to 
technical  innovations. 

Consider  first  the  comparison  of  the  MK  9  and  MK  10  sea  mines 
produced  toward  the  end  of  World  War  II  and  the  1962  -  MK  36  and  3? 
sea  mines  which  replaced  them,  and  whose  increased  lethality,  sen¬ 
sitivity,  resistance  to  countermeasures,  operating  depth,  operating 
time  on  station,  shock  resistance  to  air  drop,  and  the  like  are  due 
to  the  iWi  Events-;-"  which  have  been  documented.  Figure  U  shows  the 


Only  6?  of  those  Events  have  been  included  in  our 
statistical  analysis  in  this  preliminary  report. 
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comparative  costs  of  maintainini;  a  mined  harbor  at  a  given  level  of 
effectiveness  for  one  month.  Tliese  calculations,  incidentally,  are 
based  upon  two  real  examples.  Both  yield  the  result  that  it  would 
cost  at  least  10  times  as  much  with  the  old  mines  as  with  the  new. 

Also  the  operation  would  require  about  six  times  as  many  military 
personnel. 

Consider  as  a  second  example  the  relative  effectiveness  of  the 
195^  C-I30-A  turbo-prop  transport  compared  to  the  1964  C-l4l-A 
turbo- fan  transport.  Figure  12  shows  that  the  C-l4l-A  on  which  we 
have  documented  8l  Events,  has  a  ton-mile  cost  which  is  only  60^ 
that  of  the  C-I3O-A.  Assuming  a  10- year  operating  life,  ana  an 
operating  range  of  I9OO  miles  which  is  within  the  capability  of 
the  C-I3O-A,  we  calculate  that  the  planned  fleet  of  284  C-l4l-A's 
would  produce  a  net  savings  of  over  $4  billion  compared  to  doing 
the  same  job  with  a  fleet  of  C-130-A's.  ¥e  estimate  the  military 
manpower  needed  to  operate  the  G-141-A  fleet  would  be  about  half 
that  needed  to  operate  the  C-I3O-A  fleet. 

The  dramatic  benefits  of  technological  innovation  are  not 
always  limited  to  completely  new  designs.  As  Figure  12  also 
shews,  the  C-130's  have  been  substantially  upgraded  by  retro¬ 
fitting  and  by  modifications  based  on  1964  technology. 

Our  third  example  is  the  comparison  of  two  radars.  The  1944  SP 
and  the  1964  SPS-I(6,  whose  performance  depends  upon  86  Events.  To 
en^hasize  the  great  difference  in  performance  between  the  two  systems, 
we  shall  make  the  assumption  that  the  Fleet  requires  the  performance 
of  the  SPS-48,  but  is  forced  to  obtain  the  equivalent  performance  by 
using  SP  type  technology.  The  latter  could  be  accomplished  throu^ 
a  complicated  assembly  of  40  SP  radars  and  a  very  large  antenna. 

This  clearly  impractical  hypothetical  system  is  compared  with  the 
compact,  reliable  and  practical  SPS-48  in  Figure  I3.  It  means,  of 
course,  that  one  simply  cannot  obtain  the  SPS-48  performance  using 
SP  technology  any  more  than  the  performance  of  the  NIKE-X  radar  could 
be  obtained  with  only  the  SPS-48  technology.  In  short,  when  require¬ 
ments  are  rapidly  escalating,  new  science  and  technology  offer  the 
only  route  to  an  economic  solution. 

Suppose,  however,  that  one  had  constant  requirements.  What  would 
technology  do  in  this  case?  We  have  identified  a  recent  (1965)  search 
and  traffic  control  hei^t- finder  radar,  the  TPN-8,  whose  range  and 
altitude  coverage,  data  rate  and  sensitivity  are  very  nearly  those 
of  the  1944  SP.  We  have  not  performed  a  HINDSIPHT  analysis  on  the 
TPK-8,  but  we  believe  that  it  would  be  found  to  depend  on  a  set  of 
Events,  most  of  which  have  been  financed  by  Defense,  and  many  of 
which  probably  overlap  with  the  SPS-48  (for  one  thing,  the  two  radars 
were  developed  and  produced  by  the  same  company).  Figure  l4  shows 
the  comparison  of  the  two  radars.  We  see  that  in  each  of  the  factors — 
cost,  wei^t,  and  reliability— the  new  radar  is  five  or  more  times 
superior  to  the  SP. 
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We  have  not  attempted  to  make  an  exact  cost-effectiveness  com¬ 
parison  of  MTNOTEMAN  II  compared  to  MII'ItlTSl^IAN  I  or  of  the  MK  kb 
torpedo  compared  to  the  MK  44.  However,  if  one  were  to  try  to  do 
the  same  job  with  the  older  equipment  that  is  demanded  by  the  ad¬ 
vanced  requirements  which  are  met  by  the  newer  equipment,  we  esti¬ 
mate  that  the  costs  and  the  military  manpower  would  be  hi^er  by 
factors  similar  to  those  we  have  determined  above,  namely,  by 
factors  of  2  to  10. 

We  do  not  know  if  all  the  equipments  in  the  U.S.  weapons  and 
equipment  inventory  of  some  $80  billion  procurement  cost  are  as 
greatly  improved  by  technological  innovation  over  their  predecessors 
as  are  the  cases  which  we  have  examined.  However,  if  our  examples 
are  at  all  representative,  the  approximately  $10  billion  spent  by 
Defense  for  science  and  technology  from  19^5  to  19^3^  the  period 
during  which  98^  of  the  Events  occurred,  have  been  paid  back  many 
times  over. 

Seventh,  we  conclude  that  the  current  productivity  of  the  in-house 
laboratory  is  comparable  to  that  of  industry.  We  have  reasonably 
accurate  relevant  fiscal  information  only  for  the  period  Fl-63-65. 

For  this  Interval,  as  reported  by  the  NSF  in  the  category  of  "applied 
research",  in  "Federal  Funds  for  Science,"  the  DOD  spent  about  2>2>1o 
ol  its  money  in-house  and  about  5*+^  in  industry.  Figure  7,  describ¬ 
ing  the  relative  productivity  of  the  in-house  laboratories  demonstrates 
that  the  return  on  investment  during  this  interval  correlates  well  with 
the  level  of  investment.  The  decreasing  output  relative  to  industry  of 
the  in-house  laboratories,  over  the  20-year  period,  correlates  with  the 
relative  changes  in  professional  strength  of  the  two  categories. 
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Figure  1.  Systems  Studied 


Huaiber  available  as  of  15  June 


Figure  3.  Mark  56  and  57  Mines 


Note:  ^Includes  significant  specification  changes. 


Figure  4.  LANCE  Missile 
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Normalized  Distribu  ^on  of  All  RXD  Events 
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Figure  6.  Event  Source  Distribution 
(By  Performer) 


Figure  7.  POD  In-House  Activit 


Figure  10.  Research  Objective 
(90%  Identification) 


Figure  11.  Relative  Cost  for  Fixed  Effectiveness 
Ilk  Sd  llod  0  vs.  Mk  10  Hbd  ff  Idlbes 


Figure  12.  C-130  and  C-141  Cargo  Aircraft 
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